Papers with sensitivity analysis
Nationality Bias in Text Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that nationality biases in language models can be a factor in improving the performance of social NLP models. |
| Approach: | They propose to use a text generation model, GPT-2, to analyze how the number of internet users and the country’s economic status affects the sentiment of stories. |
| Outcome: | The proposed model accentuates biases about country-based demonyms and reduces them with the use of adversarial triggering. |
A Thorough Examination of Decoding Methods in the Era of LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Decoding methods are essential for converting language models from next-token predictors into practical task solvers. |
| Approach: | They propose to evaluate decoding methods in general-purpose large language models . they find that decoding method performance is notably task-dependent . |
| Outcome: | The proposed methods perform task-dependently and are influenced by alignment, model size, and quantization. |
Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt-based learning has been an effective paradigm for large pretrained language models (LLMs), enabling few-shot or even zero-shot learning. |
| Approach: | They propose a black-box prompt search method that clusters and prunes the search space to focus exclusively on influential prompt tokens. |
| Outcome: | The proposed method achieves state-of-the-art performance across tasks and LLMs while significantly reducing search costs. |